• Introduce _run_blocking_serialized() helper to prevent MLX/Metal race conditions
• Ensure generation lock held until worker thread completes, even on cancellation
• Replace direct asyncio.to_thread() calls with cancellation-safe wrapper
• Add regression test verifying no concurrent MLX access after request cancellation

Regression test for cancellation-safe serialization

• New regression test file with SimpleEngineCancelSerializationTests class
• Tests that cancellation does not release generation lock before worker finishes
• Verifies concurrent MLX access counter stays at 1 (no overlap)
• Uses threading events and mocks to simulate concurrent generate calls

tests/test_simple_engine_cancel_serialization.py

Implement cancellation-safe serialized blocking operations

• Add new _run_blocking_serialized() method that wraps blocking calls with cancellation safety
• Method acquires _generation_lock, runs blocking operation in thread, and waits for completion on
 cancellation
• Refactor generate() to use _run_blocking_serialized() instead of direct asyncio.to_thread()
• Refactor chat() to use _run_blocking_serialized() for both MLLM and LLM paths
• Refactor stream_chat() MLLM path to use _run_blocking_serialized() and move lock outside async
 context
• Refactor stream_generate() to use _run_blocking_serialized() for all worker calls

vllm_mlx/engine/simple.py

Description

_stream_generate_text() holds _generation_lock and now awaits _run_blocking_serialized(),
which tries to acquire the same lock again, deadlocking MLLM+MTP text-only streaming. Text-only MLLM
requests routed to this path will hang indefinitely.

Code

vllm_mlx/engine/simple.py[1199]

+            all_resps = await self._run_blocking_serialized(_run_all)

Evidence

_run_blocking_serialized() acquires _generation_lock unconditionally. _stream_generate_text()
wraps its generation body in async with self._generation_lock: and later calls `all_resps = await
self._run_blocking_serialized(_run_all)` within that block, causing nested acquisition of the same
asyncio.Lock and a deadlock.

vllm_mlx/engine/simple.py[229-246]
vllm_mlx/engine/simple.py[1012-1014]
vllm_mlx/engine/simple.py[1194-1201]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`_stream_generate_text()` currently acquires `self._generation_lock` and then calls `_run_blocking_serialized()`, which also acquires `self._generation_lock`. This deadlocks.

### Issue Context
This path is used for MLLM+MTP routing (text-only requests). The PR moved lock acquisition into `_run_blocking_serialized()`, so call sites that already hold the lock must be adjusted.

### Fix Focus Areas
- vllm_mlx/engine/simple.py[229-246]
- vllm_mlx/engine/simple.py[1012-1014]
- vllm_mlx/engine/simple.py[1194-1201]

### Implementation direction
Refactor `_stream_generate_text()` so `_run_all` (and thus MLX/Metal calls) run under `_run_blocking_serialized()` without an additional surrounding `async with self._generation_lock:` block; or add a dedicated cancellation-safe helper that does **not** reacquire the lock when the caller already holds it.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

The new regression test doesn’t assert that first_started.wait() succeeded, so it can proceed even
if the first worker never started, weakening the test and making failures/pass conditions
misleading. This can cause false positives or intermittent behavior under scheduler variance.

Code

tests/test_simple_engine_cancel_serialization.py[R59-63]

+            await asyncio.to_thread(first_started.wait, 1.0)
+
+            task1.cancel()
+            task2 = asyncio.create_task(engine.generate(prompt="second", max_tokens=8))
+

Evidence

threading.Event.wait() returns a boolean indicating whether it was set before timeout, but the
test ignores it and immediately cancels task1 and starts task2. If the first worker didn’t start
within the timeout, the test is no longer validating the intended race.

tests/test_simple_engine_cancel_serialization.py[58-63]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The test should fail fast if the first worker never started; otherwise it may test a different scenario than intended.

### Issue Context
`threading.Event.wait(timeout)` returns `True/False`. The test currently ignores this return value.

### Fix Focus Areas
- tests/test_simple_engine_cancel_serialization.py[58-63]

### Implementation direction
Capture the boolean result and assert it is `True`, e.g.:
- `started = await asyncio.to_thread(first_started.wait, 1.0)`
- `self.assertTrue(started, "first worker did not start in time")`

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

When the request is cancelled, _run_blocking_serialized() waits for the worker thread but
suppresses any worker exception without logging, making MLX/Metal failures during cancellation hard
to diagnose. This reduces observability of precisely the failure mode the PR is trying to mitigate.

Code

vllm_mlx/engine/simple.py[R240-245]

+            except asyncio.CancelledError:
+                try:
+                    await task
+                except Exception:
+                    pass
+                raise

Evidence

In the cancellation handler, exceptions from the worker task are caught with except Exception: and
dropped (pass) without logging, so a worker crash during cancellation becomes invisible in logs
while the caller only sees CancelledError.

vllm_mlx/engine/simple.py[229-245]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
If the worker raises while the outer task is cancelled, the exception is swallowed, harming debuggability.

### Issue Context
The code correctly waits for the worker to finish to keep MLX/Metal serialized, but should still emit diagnostics if the worker failed.

### Fix Focus Areas
- vllm_mlx/engine/simple.py[229-245]

### Implementation direction
In the `except asyncio.CancelledError:` block, change the bare `except Exception: pass` into a log statement (e.g., `logger.warning(..., exc_info=True)`) before re-raising the cancellation.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools